Optical device, phase plate, and image forming method

ABSTRACT

An optical device comprises a shared phase modulation mask configured to impart a first phase modulation to light of a first wavelength, and imparts a second phase modulation to light of a second wavelength, an irradiation optical system configured to cause the light of the first wavelength and the light of the second wavelength to enter the same incident region in the phase modulation mask, and a light collecting optical system configured to collect the light of the first phase-modulated first wavelength and the light of the second phase-modulated second wavelength to form an image corresponding to a point spread function.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/454,505, filed on Mar. 9, 2017, which claims priority from priorJapanese Patent Application No. 2016-047682, filed on Mar. 10, 2016,entitled, “OPTICAL DEVICE, PHASE PLATE, AND IMAGE FORMING METHOD”, theentire contents of which are incorporated herein by reference.

The present invention relates to an optical device, phase plate, andimage forming method.

BACKGROUND

In optical devices such as optical microscopes, for example, variousfunctions such as improvement of resolution and aberration correctionare realized by using phase modulation of light. Patent reference 1describes a phase modulation device that modulates phases of lightfluxes of a plurality of different wavelengths within a predeterminedbroad spectrum width. The phase modulation device diffracts the lightbeam irradiated from the light source via a diffraction grating todifferent angles for each wavelength. The light beam diffracted for eachwavelength enters the spatial phase modulation device through acondenser lens. The phase modulation device has a plurality of phasemodulation areas respectively corresponding to the respectivewavelengths, and the light flux of each wavelength is enters the phasemodulation area corresponding to each wavelength by diffraction via thediffraction grating.

SUMMARY OF THE INVENTION

In the art of Japanese Patent Application Publication No. 2010-25922,the light flux must be diffracted by a diffraction grating, a prism, orthe like in order to make the light flux enter the corresponding phasemodulation region for each wavelength. A problem arises therefore inthat the optical system becomes complex and the configuration for phasemodulation becomes complicated. The phase modulation device of patentreference 1 also gives rise to the problem of phase modulation devicebecoming larger, since the phase modulation device has a plurality ofphase modulation areas for each wavelength.

A first aspect invention relates to an optical device. The opticaldevice of this embodiment includes a shared phase modulation mask thatimparts a first phase modulation on light of a first wavelength andimparts a second phase modulation on light of a second wavelength, anirradiation optical system that causes the light of the first wavelengthand the light of the second wavelength to be incident to the sameincidence region on the phase modulation mask, and a light collectingoptical system that collects the first phase-modulated light of thefirst wavelength and the second phase-modulated light of the secondwavelength to form an image according to a point spread function.

A second aspect invention relates to an optical device. The opticaldevice of this embodiment includes a shared phase modulation mask thatimparts a phase modulation to light of a first wavelength and light of asecond wavelength, an irradiation optical system that causes the lightof the first wavelength and light of the second wavelength to beincident to the phase modulation mask, and a light collecting opticalsystem that collects the light of the first wavelength and the light ofthe second wavelength that has been phase-modulated by the phasemodulation mask to form an image according to a point spread function.The phase modulation mask is a phase plate. The phase plate has athickness between the thickness of a phase plate for the light of thefirst wavelength and the thickness of a phase plate for the light of thesecond wavelength.

A third aspect invention relates to an optical device. The opticaldevice of this embodiment includes a shared phase modulation mask thatimparts a phase modulation to light of a first wavelength and light of asecond wavelength, an irradiation optical system that causes the lightof the first wavelength and light of the second wavelength to beincident to the phase modulation mask, and a light collecting opticalsystem that collects the light of the first wavelength and the light ofthe second wavelength that has been phase-modulated by the phasemodulation mask to form an image according to a point spread function.The phase modulation mask is a phase modulation device capable ofsetting a phase modulation pattern based on an input. The phasemodulation device applies phase modulation to the light of the firstwavelength and the light of the second wavelength by a phase modulationpattern set based on an input gradient between the gradient of the lightof the first wavelength and the gradient of the light of the secondwavelength.

A fourth aspect of the invention relates to a phase plate that causeslight of a first wavelength and light of a second wavelength to beincident on the same incidence region. The phase plate of thisembodiment includes a first region configured to apply a first phasemodulation to light of the first wavelength, and a second regionconfigured to apply a second phase modulation to light of the secondwavelength in the incidence area, and the phase plate is configured toapply a first phase modulation on the light of the first wavelength andapply a second phase modulation on the light of the second wavelength.

A fifth aspect of the invention relates to a method for forming an imageaccording to a point spread function from light of a first wavelengthand light of a second wavelength. The image forming method of thisembodiment causes light of a first wavelength and light of a secondwavelength to be incident to the same incidence region in a shared phasemodulation mask that imparts a first phase modulation on the light ofthe first wavelength and imparts a second phase modulation on the lightof the second wavelength, uses the shared phase modulation mask toimpart phase modulation to light of a first wavelength and light of thesecond wavelength and collects the first phase-modulated light of thefirst wavelength and the second phase-modulated light of the secondwavelength to form an image according to a point spread function.

A sixth aspect of the invention relates to a method for forming an imageaccording to a point spread function from light of a first wavelengthand light of a second wavelength. The image forming method of thisembodiment causes the light of the first wavelength and the light of thesecond wavelength to be incident on a shared phase plate that impartsphase modulation to the light of the first wavelength and the light ofthe second wavelength, the phase plate having a thickness between thethickness of a phase plate used for the light of the first wavelengthand the thickness of a phase plate used for the light of the secondwavelength, and collects the light of the first wavelength and the lightof the second wavelength that has been modulated by the phase plate toform an image according to a point spread function.

A seventh aspect of the invention relates to a method for forming animage according to a point spread function from light of a firstwavelength and light of a second wavelength. The image forming method ofthis embodiment causing the light of the first wavelength and the lightof the second wavelength to be incident to a shared phase modulationdevice that imparts phase modulation to the light of the firstwavelength and the light of the second wavelength by a phase modulationpattern set based by an input of a gradation between the gradation forthe light of the first wavelength and the gradation for the light of thesecond wavelength, collects the light of the first wavelength and thelight of the second wavelength that has been phase modulated by thephase modulation device to form an image according to a point spreadfunction.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view showing the structure of the optical deviceprovided with a phase modulation mask of the embodiment;

FIG. 2 (a) is a schematic view showing two bright spot images rotated onthe imaging plane according to the position of the emission point offluorescence in the Z axis direction of the embodiment; FIG. 2 (b) showsthe DH-PSF of the embodiment;

FIG. 3 (a) is a schematic view showing the active state of all firstfluorescence dye of the embodiment; FIG. 3 (b) is a schematic viewshowing the inactive state of all first fluorescence dye of theembodiment; FIGS. 3 (c) and 3 (d) are schematic views showing the activestate of part of the first fluorescence dye of the embodiment; FIG. 3(e) is a schematic view showing the active state of all secondfluorescence dye of the embodiment; FIG. 3 (f) is a schematic viewshowing the inactive state of all second fluorescence dye of theembodiment; FIGS. 3 (g) and 3 (h) are schematic views showing the activestate of part of the second fluorescence dye of the embodiment;

FIG. 4 (a) through 4 (e) are schematic views showing the sequence ofobtaining a first 3-dimensional super-resolution image of theembodiment; FIG. 4 (f) is a schematic view showing the sequence ofobtaining the number of a first substance of the embodiment;

FIG. 5 (a) is a schematic view showing a first 3-dimensionalsuper-resolution image obtained from a plurality of first 2-dimensionalimages, and a second 3-dimensional super-resolution image obtained froma plurality of second 2-dimensional images of the embodiment; FIG. 5 (b)is a schematic view showing the first and second 3-dimensionalsuper-resolution images obtained from a plurality of common2-dimensional images in a modification of the embodiment;

FIG. 6 (a) is a schematic view showing the structure of the opticaldevice provided with a phase modulation mask of the embodiment; FIG. 6(b) is a schematic view showing the structure of the phase modulationdevice of the embodiment;

FIGS. 7 (a) and 7 (b) show the first phase modulation pattern and secondphase modulation pattern set in the phase modulation device of theembodiment;

FIGS. 8 (a) and 8 (b) show the imaging state and graph when the firstfluorescent light is observed using the first phase modulation patternof the embodiment; FIGS. 8 (c) and 8 (d) show the imaging state andgraph when the second fluorescent light is observed using the firstphase modulation pattern of the embodiment; FIGS. 8 (e) and 8 (f) showthe imaging state and graph when the first fluorescent light is observedusing the second phase modulation pattern of the embodiment; FIGS. 8 (g)and 8 (h) show the imaging state and graph when the second fluorescentlight is observed using the second phase modulation pattern of theembodiment;

FIG. 9 shows the structure of a phase modulation pattern of example 1;

FIG. 10 shows the imaging state of the first fluorescent light using thephase modulation pattern of example 1;

FIG. 11 shows the imaging state of the second fluorescent light usingthe phase modulation pattern of example 1;

FIGS. 12 (a) and 12 (b) show the imaging state and graph when the firstfluorescent light is observed using the first phase modulation patternof example 1;

FIGS. 12 (c) and 12 (d) show the imaging state and graph when the secondfluorescent light is observed using the first phase modulation patternof example 1;

FIG. 13 (a) through 13 (c) illustrate details of the integration of thefirst phase modulation pattern and second phase modulation patternaccording to the phase modulation pattern of example 1; FIG. 13 (a)through 13 (c) schematically show one pixel position on a liquid crystalpanel in the first phase modulation pattern, second phase modulationpattern, and the phase modulation pattern of example 1, respectively;

FIG. 14 (a) is a schematic diagram showing the arrangement of regions inthe phase modulation pattern of example 2; FIG. 14 (b) through 14 (d)schematic diagrams showing the structure of the phase modulation patternof example 2;

FIGS. 15 (a) and 15 (b) show the imaging state and graph when the firstfluorescent light is observed using the phase modulation pattern ofexample 2;

FIGS. 15 (c) and 15 (d) show the imaging state and graph when the secondfluorescent light is observed using the phase modulation pattern ofexample 2;

FIGS. 15 (e) and 15 (f) show the imaging state and graph when the firstfluorescent light and second fluorescent light is observed using thephase modulation pattern of example 2;

FIGS. 16 (a) and 16 (b) show the result of observing the firstfluorescent light and the second fluorescent light with a wide field ofview using the phase modulation pattern of example 2;

FIG. 17 (a) is a schematic diagram showing the arrangement of regions inthe phase modulation pattern of example 3; FIG. 17 (b) through 17 (d)schematic diagrams showing the structure of the phase modulation patternof example 3;

FIGS. 18 (a) and 18 (b) show the imaging state and graph when the firstfluorescent light is observed using the phase modulation pattern ofexample 3;

FIGS. 18 (c) and 18 (d) show the imaging state and graph when the secondfluorescent light is observed using the phase modulation pattern ofexample 3;

FIGS. 18 (e) and 18 (f) show the imaging state and graph when the firstfluorescent light and second fluorescent light is observed using thephase modulation pattern of example 3;

FIGS. 19 (a) and 19 (b) respectively show the result of observing thefirst fluorescent light and the second fluorescent light with a widefield of view using the phase modulation pattern of example 3;

FIG. 20 (a) is a schematic diagram showing the arrangement of regions inthe phase modulation pattern of example 3; FIG. 20 (b) through 20 (d)are schematic diagrams showing the structure of the phase modulationpattern of example 4;

FIGS. 21 (a) and 21 (b) show the imaging state and graph when the firstfluorescent light is observed using the phase modulation pattern ofexample 4;

FIGS. 21 (c) and 21 (d) show the imaging state and graph when the secondfluorescent light is observed using the phase modulation pattern ofexample 4;

FIGS. 21 (e) and 21 (f) show the imaging state and graph when the firstfluorescent light and second fluorescent light is observed using thephase modulation pattern of example 4;

FIGS. 22 (a) and 22 (b) show the result of observing the firstfluorescent light and the second fluorescent light with a wide field ofview using the phase modulation pattern of example 4;

FIG. 23 (a) is a schematic diagram showing the arrangement of regions inthe phase modulation pattern of example 5; FIG. 23 (b) through 23 (e)are schematic diagrams showing the structure of the phase modulationpattern of example 5;

FIGS. 24 (a) and 24 (b) show the imaging state and graph when the firstfluorescent light is observed using the phase modulation pattern ofexample 5;

FIGS. 24 (c) and 24 (d) show the imaging state and graph when the secondfluorescent light is observed using the phase modulation pattern ofexample 5; FIGS. 24 (e) and 24 (g) show the imaging state and graph whenthe first fluorescent light is observed using the phase modulationpattern of example 5; FIGS. 24 (f) and 24 (h) show the imaging state andgraph when the second fluorescent light is observed using the phasemodulation pattern of example 5;

FIGS. 25 (a) and 25 (b) show the result of observing the firstfluorescent light and the second fluorescent light with a wide field ofview using the phase modulation pattern of example 5;

FIG. 26 shows the structure of a phase modulation pattern of example 6;

FIGS. 27 (a), 27 (c), 27 (e), and 27 (g) show the imaging state when thefirst fluorescent light is observed using the phase modulation patternof example 6; FIGS. 27 (b), 27 (d), 27 (f), and 27 (h) show the imagingstate when the second fluorescent light is observed using the phasemodulation pattern of example 6;

FIGS. 28 (a), 28 (c), 28 (e), and 28 (g) show the imaging state when thefirst fluorescent light is observed using the phase modulation patternof example 6; FIGS. 28 (b), 28 (d), 28 (f), and 28 (h) show the imagingstate when the second fluorescent light is observed using the phasemodulation pattern of example 6;

FIG. 29 (a) is a schematic view showing the phase plate manufactured soas to correspond to the first phase modulation pattern of theembodiment;

FIG. 29 (b) is a schematic view showing the phase plate manufactured soas to correspond to the second phase modulation pattern of theembodiment; FIG. 29 (c) a schematic view showing a phase plateintegrated as in the phase modulation pattern of example 1; FIG. 29 (d)through 29 (f) are schematic views showing cross sections when the phaseplate according to the embodiment is sectioned along a plane parallel tothe thickness direction;

FIG. 30 (a) is a schematic view showing the phase plate manufactured soas to correspond to the first phase modulation pattern of theembodiment; FIG. 30 (b) is a schematic view showing the phase platemanufactured so as to correspond to the second phase modulation patternof the embodiment; FIG. 30 (c) a schematic view showing a phase plateintegrated as in the phase modulation pattern of example 2; FIG. 30 (d)through 30 (f) schematically shows regions when viewing in the thicknessdirection of the phase plate of the embodiment; and FIG. 30 (g) through30 (i) are schematic views showing cross sections when the phase plateaccording to the embodiment is sectioned along a plane parallel to thethickness direction.

DESCRIPTION OF THE EMBODIMENTS OF THE INVENTION

The following embodiments apply the invention to optical devices forobserving two types of fluorescent light with different centerwavelengths. The optical device of the embodiment is a fluorescencemicroscope that irradiates light on a sample and captures thefluorescent light given off from the sample via an imaging part. Theoptical device to which the invention can be applied is not limited tothe following embodiments, and may be a microscope other than afluorescence microscope, that is, an imaging device such as a camera, atelescope, an endoscope, a planetarium or the like. The optical deviceto which the invention can be applied also is not limited to a devicefor imaging and observing fluorescent light, and may be a device forimaging and observing light other than fluorescent light.

As shown in FIG. 1 , the optical device 10 is provided with anirradiating optical system 30, beam expander 37, stage 40, phasemodulation mask 50, light collecting optical system 60, controllers 71and 72, and information processing device 100. Mutually orthogonal XYZaxes are shown in FIG. 1 .

A slide glass 41 on which a sample is placed is installed on the stage40. The sample of the embodiment is a biological sample containing testcells. The test cells, for example, are collected from diseased tissue.The nucleus of the test cell contains a first substance and a secondsubstance. The first substance and the second substance to be imagedare, for example, biological substances such as genes, proteins orpeptides that may be disease markers. The first substance of theembodiment is the HER2 gene, and the second substance of the embodimentis CEP17 which is a centromere region of chromosome 17.

Fluorescent substances are bound to the first substance and the secondsubstance, respectively, when preparing the sample. In the embodiment,the fluorescent substance bound to the first substance is a firstfluorescent dye, and the fluorescent substance bound to the secondsubstance is a second fluorescent dye. The nucleus of the test cellsalso are specifically stained by a third fluorescent dye when preparingthe sample.

The first fluorescent dye can be switched between an active state inwhich a first fluorescent light having a center wavelength of a firstwavelength is given off when irradiated with light from a light source21 (described later), and an inactive state in which the firstfluorescent light is not given off even when irradiated by light fromthe light source 21. The first fluorescent dye is inactivated whenirradiated with light from the light source 21, and is activated whenlight from a light source 23 to be described later. The secondfluorescent dye can be switched between an active state in which asecond fluorescent light having a center wavelength of a secondwavelength is given off when irradiated with light from a light source22 (described later), and an inactive state in which the secondfluorescent light is not given off even when irradiated by light fromthe light source 22. The second fluorescent dye is inactivated whenirradiated with light from the light source 22, and is activated whenlight from the light source 23. The third fluorescent dye gives off athird fluorescent light having a center wavelength of a third wavelengthwhen irradiated by light from the light source 23.

The substance to be imaged is not limited to the fluorescent dye thatbinds to the substance as described above, and may be a substance thatproduces autofluorescence. The sample placed on the slide glass 41 isnot limited to a biological sample. The substance to be imaged is notlimited to a substance contained in a biological sample, and may be asubstance not derived from the biological sample. For example, thesubstance to be imaged may be fluorescent beads or fluorescentsubstances such as fluorescent particles and the like.

The irradiation optical system 30 includes a light source section 20, ashutter 31, a quarter-wave plate 32, a beam expander 33, a condenserlens 34, a dichroic mirror 35, and an objective lens 36. The irradiationoptical system 30 irradiates the sample with light to generate firstthrough third fluorescent lights from the fluorescently labeled firstthrough third substances contained in the sample, respectively, andcauses the first fluorescent light and the second fluorescent light tobe incident on the same incident region in the phase modulation mask 50.

The light source section 20 includes light sources 21 through 23, mirror24, and dichroic mirrors 25 and 26.

The light sources 21 through 23 respectively emit light of differentwavelengths. Specifically, the wavelengths of the light emitted from thelight sources 21, 22 and 23 are 640 nm, 488 nm, and 405 nm,respectively. A laser light source is preferably used, but a mercurylamp, a xenon lamp, an LED, or the like may be used as the light sources21 through 23. As described above, the light emitted from the lightsource 21 excites the first fluorescent dye contained in the test cellto generate the first fluorescent light, and inactivates the firstfluorescent dye. The light emitted from the light source 22 excites thesecond fluorescent dye contained in the test cell to generate the secondfluorescent light, and inactivates the second fluorescent dye. The lightemitted from the light source 23 excites the third fluorescent dyecontained in the test cell to generate the third fluorescent light, andinactivates the third fluorescent dye. Note that, in the embodiment, thefirst wavelength that is the center wavelength of the first fluorescentlight is 690 nm, and the second wavelength that is the center wavelengthof the second fluorescent light is 530 nm.

The mirror 24 reflects the light from the light source 21. The dichroicmirror 25 transmits the light from the light source 21 and reflects thelight from the light source 22. The dichroic mirror 26 transmits thelight from the light sources 21 and 22, and reflects the light from thelight source 23. The optical axes of the light from the light sources 21through 23 are mutually matched by the mirror 24 and dichroic mirrors 25and 26. Note that one light source may emit light having wavelengths of640 nm, 488 nm, and 405 nm instead of the light sources 21 through 23.

The shutter 31 is driven by the controller 71 and switches between astate of allowing light emitted from the light source section 20 to passthrough and a state of blocking light emitted from the light sourcesection 20. In this way the irradiation time of light on the test cellis adjusted. The quarter-wave plate 32 converts the linearly polarizedlight emitted from the light source section 20 into circularly polarizedlight. The fluorescent dye reacts to light of a predeterminedpolarization direction. Therefore, the polarization direction of theexcitation light is easily matched to the polarization direction inwhich the fluorescent dye reacts by converting the excitation lightemitted from the light source section 20 into circularly polarizedlight. In this way it is possible to excite fluorescence efficiently inthe fluorescent dye contained in the test cell. The beam expander 33expands the irradiation area of the light on the slide glass 41. Thecondenser lens 34 condenses the light so that parallel light is emittedfrom the objective lens 36 to the slide glass 41.

Dichroic mirror 35 reflects the light emitted from the light sourcesection 20, and transmits fluorescent light given off from the testcell. The objective lens 36 guides the light reflected by the dichroicmirror 35 to the slide glass 41. The stage 40 is driven by thecontroller 72 and moves on the horizontal plane within the λ-Y plane.The fluorescent light given off from the test cells passes through theobjective lens 36, passes through the dichroic mirror 35, and isrendered to parallel light by the beam expander 37.

The phase modulation mask 50 provides phase modulation for the firstfluorescent light and the second fluorescent light. The phase modulationmask 50 is disposed on the Fourier plane of the optical system formed bythe objective lens 36, the dichroic mirror 35, the beam expander 37, andthe condenser lens 61, and has the effect of modulating the phase of thelight incident on the same incident region of the phase modulation mask50.

Note that when the phase difference is “θ”, and when the phasedifference is “θ+n×2π” (where n=±0, 1, 2, 3 . . . ), the phasedifference given to the fluorescent light is substantially the same.Therefore, the phase difference given to the first fluorescent light andthe phase difference given to the second fluorescent light by the phasemodulation mask 50 is not limited to a single value, and also may be avalue obtained by adding n×2π to the single value.

The phase modulation mask 50 is arranged as shown in FIG. 1 whenmodulating the phase of the light passing through the phase modulationmask 50. The phase modulation mask 50 that modulates the phase of thetransmitted light is configured by, for example, a phase plate made of atransparent member such as an acrylic resin, a phase modulation devicewith a liquid crystal panel or the like.

The phase modulation mask 50 is arranged as shown in FIG. 6 (a) whenmodulating the phase of the light reflected by the phase modulation mask50. The phase modulation mask 50 that modulates the phase of thereflected light is configured by, for example, a phase modulation devicewith a liquid crystal panel, a deformable mirror, a reflecting memberconfigured similarly to a deformable mirror and the like. Note that whenthe phase modulation mask 50 is configured by a phase modulation devicehaving a liquid crystal panel, a polarizing plate 38 and a mirror 39 arearranged instead of the beam expander 37, as shown in FIG. 6 (a), andthe mask 50 is arranged at the position of the phase modulation device51. When the phase modulation mask 50 is configured by a deformablemirror, a reflecting member constructed similarly to a deformable mirroror the like, the polarizing plate 38 is omitted from the configurationshown in FIG. 6 (a), and the phase modulation mask 50 is arranged at theposition of the phase modulation device 51.

A configuration example of the phase modulation mask 50 that modulatesthe phase of the transmitted light will be described later withreference to FIGS. 29 (a) to 30 (i). A configuration example of thephase modulation mask 50 that modulates the phase of the reflected lightwill be described later with reference to FIGS. 6 (a) to 28 (h).

The phase modulation mask 50 forms an image according to the pointspread function of the first fluorescent light given off from the firstfluorescent dye, and forms an image according to the point spreadfunction of the second fluorescent light given off from the secondfluorescent dye. In the phase modulation mask 50 of the embodiment, thefirst fluorescent light given off from one first fluorescent dye isimaged at two focal points on the imaging surface 62 a of the imagingpart 62, and the second fluorescent light given off from one secondfluorescent dye is imaged at two focal points on the imaging surface 62a of the imaging unit 62. Such a point spread function is called DH-PSF(Double-Helix Point Spread Function). The phase modulation mask 50modulates the phases of the first fluorescent light and the secondfluorescent light entering the Fourier plane, and forms an imagecorresponding to the DH-PSF on the imaging surface 62 a for both thefirst fluorescent light and the second fluorescent light.

Note that the phase modulation mask 50 is not configured to form animage according to the DH-PSF on the imaging surface 62 a for the thirdfluorescent light, unlike the case of the first fluorescent light andthe second fluorescent like. The reason for this is that, as will bedescribed later, the third fluorescent light is used only foridentifying the region of the nucleus. Although the phase of the thirdfluorescent light transmitted through the phase modulation mask 50 ismodulated somewhat by the phase modulation mask 50, it is possible tosufficiently identify the region of the nucleus if the third fluorescentlight is imaged by the imaging part 62.

The flight collecting optical system 60 condenses the phase-modulatedfirst fluorescent light and the second fluorescent light to form animage corresponding to the DH-PSF. The light collecting optical system60 includes a collective lens 61 and imaging part 62. The collectivelens 61 collects the fluorescent light that passes through the phasemodulation mask 50, and guides the fluorescent light to the imagingsurface 62 a of the imaging part 62. The imaging part 62 captures animage of the fluorescent light irradiated on the imaging surface 62 a,and generates a two-dimensional image. The imaging part 62 is configuredby, for example, a CCD or the like.

Here, as described above, the first fluorescent light given off from onefirst fluorescent dye and the second fluorescent light given off fromone second fluorescent dye are formed at to focal points on the imagingsurface 62 a via the function of the phase modulation mask 50. That is,an image corresponding to the DH-PSF of the first fluorescent light andthe second fluorescent light is formed on the imaging surface 62 a. Atthis time, due to the action of the phase modulation mask 50, the brightspot images respectively corresponding to the two focal points areshifted in the Z axis direction, that is, rotated and formed on theimaging surface 62 a according to position of the fluorescent lightemitting point in the depth direction of the slide glass 41, as shown inFIG. 2 (a), through the action of the phase modulation mask 50. That is,the angle formed by the line connecting the two bright spot images andthe reference line changes on the imaging surface 62 a according to theposition of the emission point of the fluorescent light in the Z axisdirection.

That is, the phase modulation mask 50 is configured to modulate thephase of the first fluorescent light so as to form a DH-PSF in which twobright spot images of the first fluorescent light rotate on the imagingsurface 62 a according to the distance between the objective lens 36 andthe first fluorescent dye in the sample. Similarly, the phase modulationmask 50 is configured to modulate the phase of the second fluorescentlight so as to form a DH-PSF in which two bright spot images of thesecond fluorescent light rotate on the imaging surface 62 a according tothe distance between the objective lens 36 and the second fluorescentdye in the sample.

For example, the fluorescent light given off from the fluorescent dyesat two different positions in the Z-axis direction on the slide glass 41is divided into two by the phase modulation mask 50 and irradiated ontothe imaging surface 62 a. At this time, the straight line connecting thetwo bright spot images on the imaging surface 62 a forms an angle of +θ1with the reference line for one of the fluorescent dyes, for example,and forms an angle of +θ2 with the reference line relative to the otherfluorescent dye, as shown in FIG. 2 (a). Therefore, if a straight lineconnecting the two bright spot images acquires an angle formed withrespect to the reference line, the position of the fluorescent dye inthe Z axis direction can be acquired. In the embodiment, regarding thefirst fluorescent light and the second fluorescent light, the positionin the Z-axis direction is acquired as described above based on thetwo-dimensional image captured by the imaging part 62.

Note that DH-PSF can be represented by the equation shown in FIG. 2 (b).In the equation of FIG. 2 (b), ‘bright spot 1’ and ‘bright spot 2’represents the two bright spot images formed on the imaging surface 62a, as shown in FIG. 2 (a). ‘Coordinates on the imaging surface’represent the coordinates on the imaging surface 62 a of the fluorescentdyes that are the source of the two bright spot images.

As described above, both the first fluorescent light and the secondfluorescent light are incident on the same incident area of the phasemodulation mask 50, and the phase modulation mask 50 respectivelymodulates the phase of the first fluorescent light and the secondfluorescent light which have mutually different wavelengths. In thisway, it is not necessary to guide the first fluorescent light to aregion for modulating the phase of the first fluorescent light, or toguide the second fluorescent light to a region for modulating the phaseof the second fluorescent light. Therefore, since there is no need toseparately provide the phase modulation area of the first fluorescentlight and the phase modulation area of the second fluorescent light,there is no need for a diffraction grating to diffract the flux or aprism to divide the flux in order to guide the first fluorescent lightand the second fluorescent light to separate phase modulation areas.Therefore, according to the phase modulation mask 50, the phase can bemodulated with respect to both the first fluorescent light and thesecond fluorescent light with a simple configuration, and an imagecorresponding to the DH-PSF of the first fluorescent light and thesecond fluorescent light can be formed.

When the first fluorescent light and the second fluorescent light aresplit into different optical paths, there also is a possibility that animage corresponding to the desired DH-PSF can not be formed due tomisalignment during the assembly of the optical elements of therespective optical paths. However, according to the phase modulationmask 50, it is possible to suppress the influence caused by misalignmentduring the assembly of the optical elements the like since an opticalelement for branching the optical path becomes unnecessary. Therefore,it is possible to generate a highly accurate two-dimensional image. Suchimprovement of the accuracy of the two-dimensional image is particularlydesirable in the optical device 10 of the embodiment for generating athree-dimensional super-resolution image which will be described later.

Returning to FIG. 1 , the information processing device 100 is apersonal computer that includes a body 110, display part 120, and inputpart 130. The body 110 includes a processing part 111, memory part 112,and interface 113.

The processing part 111, for example, may be configured by a CPU. Thememory part 112, for example, may be configured by a ROM, RAM, hard diskor the like. The processing part 111 controls the light sources 21through 23 of the light source section 20, imaging part 62, andcontroller 71 and 72 through the interface 113 based on a program storedin the memory part 112.

The processing part 111 acquires the position in the Z-axis direction ofthe light emission point of the first fluorescent light as describedabove based on the two-dimensional image of the first fluorescent lightto generate a three-dimensional super-resolution image of the firstfluorescent light. Similarly, the processing part 111 acquires theposition in the Z-axis direction of the light emission point of thesecond fluorescent light as described above based on the two-dimensionalimage of the second fluorescent light to generate a three-dimensionalsuper-resolution image of the second fluorescent light. Hereinafter, thetwo-dimensional image of the first fluorescent light is referred to as a“first two-dimensional image”, the two-dimensional image of the secondfluorescent light is referred to as a “second two-dimensional image”,and the two-dimensional image of the third fluorescent light is referredto as the “third two-dimensional image”. The three-dimensionalsuper-resolution image of the first fluorescent light is referred to asa “first three-dimensional super-resolution image” and thethree-dimensional super-resolution image of the second fluorescent lightis referred to as a “second three-dimensional super-resolution image”.

The display part 120 is a display for showing the processing results andthe like of the processing part 111. The input part 130 is a mouse andkeyboard for receiving input instructions from a user.

Next, the generation procedure of the first and second three-dimensionalsuper-resolution images will be described.

First, the procedure for acquiring the first two-dimensional image willbe described referring to FIGS. 3 (a) to 3 (d).

As shown in FIG. 3 (a), In the preparation of the sample, the firstfluorescent dye is bound to the first substance via an intermediatesubstance that specifically binds to the first substance. Since thefirst substance is a gene, a nucleic acid probe can be used as anintermediate substance. A plurality of first fluorescent dyes is boundto one first substance. FIG. 3 (a) schematically shows two firstsubstances to which the first fluorescent dye is bound, respectively. Inthe initial state, all the first fluorescent dyes are active. When thetest cell is irradiated with light from the light source 21 for apredetermined time in the state shown in FIG. 3 (a), all the firstfluorescent dyes become inactive as shown in FIG. 3 (b).

When the test cell is irradiated with light from the light source 23 fora predetermined time in the state shown in FIG. 3 (a), some of the firstfluorescent dyes become active as shown in FIG. 3 (c). The ratio of thefirst fluorescent dye to be activated changes by adjusting theirradiation time of light from the light source 23. When the test cellis irradiated with the light from the light source 21 again for apredetermined time in the state shown in FIG. 3 (c), the firstfluorescent light is given off from the activated first fluorescent dye,then all the first fluorescent dyes enter the inactive state as shown inFIG. 3 (b).

When the test cell is irradiated with light from the light source 23again for a predetermined time, some of the first fluorescent dyesbecome active as shown in FIG. 3 (d), for example. When the test cell isirradiated with the light from the light source 21 again for apredetermined time in the state shown in FIG. 3 (d), the firstfluorescent light is given off from the activated first fluorescent dye,then all the first fluorescent dyes enter the inactive state as shown inFIG. 3 (b). As shown in FIGS. 3 (c) and 3 (d), the distribution of thefirst fluorescent dye activated in each activation process is differenteach time.

The processing part 111 drives the light sources 21 and 23 to repeatedlyactivate and deactivate the first fluorescent dye as described above.The imaging part 62 images the distribution of the first fluorescent dyewhich is different each time. In this way, the processing part 111acquires a plurality of first two-dimensional images, for example, 3000first two-dimensional images.

The procedure for acquiring the second two-dimensional image will bedescribed below with reference to FIGS. 3 (e) to 3 (h). The acquisitionof the second two-dimensional image is performed in substantially thesame manner as acquisition of the first two-dimensional image.

When the test cell is irradiated with light from the light source 22 fora predetermined time in the initial state shown in FIG. 3 (e), all thesecond fluorescent dyes become inactive as shown in FIG. 3 (f). When thetest cell is irradiated with light from the light source 23 for apredetermined time in the state shown in FIG. 3 (f), some of the secondfluorescent dyes become active as shown in FIG. 3 (g). When the testcell is irradiated with the light from the light source 22 again for apredetermined time in the state shown in FIG. 3 (g), the secondfluorescent light is given off from the activated second fluorescentdye, then all the second fluorescent dyes become inactive state as shownin FIG. 3 (f). When the test cell is irradiated with light from thelight source 23 again for a predetermined time, some of the secondfluorescent dyes become active as shown in FIG. 3 (h), for example.

The processing part 111 drives the light sources 22 and 23 to repeatedlyactivate and deactivate the second fluorescent dye as described above.The imaging part 62 images the distribution of the second fluorescentdye which is different each time. In this way, the processing part 111acquires a plurality of second two-dimensional images, for example, 3000first two-dimensional images.

The procedure for generating the first three-dimensionalsuper-resolution image will be described below with reference to FIGS. 4(a) to 4 (e). Note that since the procedure of generating the secondthree-dimensional super-resolution image is the same as that of thefirst three-dimensional super-resolved image, only the procedure ofgenerating the first three-dimensional super-resolved image will bedescribed below.

As shown in FIG. 4 (a), a plurality of first two-dimensional images isacquired as described above. On the first two-dimensional image shown inFIG. 4 (a), the captured first fluorescent light is indicated by blackcircles. As shown in FIG. 4 (b), the processing part 111 extracts thefirst fluorescent bright spot 81 by Gaussian fitting in each of thefirst two-dimensional images. Then, the processing part 111 acquires thecoordinates and brightness in the λ-Y plane with respect to theextracted bright spot 81.

Subsequently, the processing part 111 refers to two bright spots 81having similar brightness at a distance within a predetermined range.The processing part 111 causes the referenced to two bright spots 81 tobe fitted with the templates of the two bright spots stored in advancein the storage unit 112. The processing part 111 pairs the two brightspots 81 that can be fitted with a certain accuracy or higher, assumingthat the first fluorescent light given off from one first fluorescentdye is divided by the phase modulation mask 50.

Then, as shown in FIG. 4 (c), the processing part 111 obtains a point 82on the X-Y plane of the first fluorescent dye which is the source of thetwo bright spots 81 based on the pair of two bright spots 81.Subsequently, as shown in FIG. 4 (d), the processing part 111 acquiresan angle θ between the reference line and a straight line connecting thepair of two bright spots 81. The processing part 111 calculates thecoordinates of the first fluorescent dye in the Z-axis direction basedon the acquired angle θ. In this manner, as shown in FIG. 4 (e), theprocessing part 111 acquires the three-dimensional coordinates of theplurality of first fluorescent dyes based on the coordinates in the X-Yplane and the coordinates in the Z axis direction. Then, as shown inFIG. 4 (e), the processing part 111 generates a first three-dimensionalsuper-resolution image by superimposing the plurality ofthree-dimensional coordinates acquired in each of the firsttwo-dimensional images.

In this manner, when the first and second three-dimensionalsuper-resolution images are acquired, the light emission points of thefirst fluorescent light and the light emission points of the secondfluorescent light can be accurately grasped compared to when the firstand second two-dimensional images are used. As a result, a physician orthe like can comprehend the distribution of the first substance in theZ-axis direction with reference to the first and secondthree-dimensional super-resolution images, and can more appropriatelydetermine the disease status and the treatment policy.

Next, the procedure for acquiring the number of first substances will bedescribed referring to FIG. 4 (f).

As shown in FIG. 4 (f), the processing part 111 classifies thecoordinate points of the first three-dimensional super-resolution imageinto groups corresponding to the first substance. For example, theprocessing part 111 scans a predetermined reference space in athree-dimensional coordinate space, and determines whether the number ofcoordinate points included in the reference space is larger than athreshold value, and extracts the position of the reference space wherethe number of coordinate points is larger than the surroundings. Then,the processing part 111 classifies the group of the coordinate pointsincluded in the reference space at the extracted position into a groupcorresponding to one first substance as indicated by a broken line inFIG. 4 (f).

Subsequently, the processing part 111 acquires the range of the nucleusin the three-dimensional space of the test cell. Specifically, theprocessing part 111 displaces the objective lens 36 in the Z-axisdirection to acquire a third two-dimensional image based on the thirdfluorescent light at a plurality of different focus positions in theZ-axis direction. In the third two-dimensional image, the region inwhich the third fluorescent light is detected corresponds to thenucleus, and the region in which the third fluorescent light is notdetected corresponds to outside the nucleus, that is, cytoplasm and thelike. For each of the plurality of third two-dimensional images, theprocessing part 111 acquires the outline of the nucleus from the areawhere the third fluorescent light is detected. Then, the processing part111 acquires the nucleus range in the three-dimensional coordinate spacebased on each focus position and the outline of the nucleus at theposition.

Subsequently, the processing part 111 acquires the number of groupsincluded in the nucleus range of the test cells in the three-dimensionalcoordinate space as the number of the first substance. Note that when aplurality of test cells are included in the first three-dimensionalsuper-resolution image, the number of first substances can bedetermined, for example, by averaging the number of first substancesacquired for each test cell. The processing part 111 similarly acquiresthe number of second substances based on the second three-dimensionalsuper-resolution image.

The processing part 111 calculates the ratio of the number of the firstsubstances and the number of the second substances acquired as describedabove, that is, “the number of the first substances/the number of thesecond substances”. The ratio of the “number of first substances/numberof second substances”, for example, can be judged to be positive forbreast cancer if it is larger than 2.2, negative for breast cancer ifless than 1.8, and the boundary can be judged as 1.8 or more to 2.2 orless.

As described above, when the numbers of the first substance and thesecond substance are acquired based on the first and secondthree-dimensional images, “the number of the first substance/the numberof the second substance” can be calculated with high accuracy. As aresult, a judgment result with higher accuracy can be presented to aphysician or the like.

Modification Example of Imaging Procedure

In the imaging procedure, the first fluorescent light and the secondfluorescent light were captured separately by the imaging part 62. Inthis case, as shown in FIG. 5 (a), the processing part 111 acquires aplurality of second two-dimensional images after acquiring a pluralityof first two-dimensional images. Then, the processing part 111 acquiresthe first three-dimensional super-resolution image based on theplurality of first two-dimensional images, and acquires the secondthree-dimensional super-resolution image based on the plurality ofsecond two-dimensional images. In this way, when the firsttwo-dimensional image and the second two-dimensional image are capturedseparately, the time required for capturing is lengthened.

In the imaging procedure, the first fluorescent light and the secondfluorescent light also may be captured simultaneously by the imagingpart 62. In this case, when the first and second fluorescent dyes are inthe active state, the processing part 111 turns on the light sources 21and 22 at the same time, and simultaneously irradiates the test cellswith light from the light sources 21 and 22. In this way, the firstfluorescent light and the second fluorescent light are given offsimultaneously from the test cells, and the first fluorescent light andthe second fluorescent light are simultaneously irradiated on theimaging surface 62 a of the imaging part 62. As shown in FIG. 5 (b), thetwo-dimensional image acquired at this time is a common two-dimensionalimage in which the first two-dimensional image and the secondtwo-dimensional image are superimposed. Note that in the case in whichthe first fluorescent light and the second fluorescent light aregenerated at the same time, the imaging part 62 is configured with acolor CCD or the like.

Also in this case, as shown in FIG. 5 (b), the processing part 111generates a first three-dimensional super-resolution image based on thefirst fluorescent light in the common two-dimensional image, andgenerates a second three-dimensional super-resolution image based on thesecond fluorescent light in the common two-dimensional image. When thefirst fluorescent light and the second fluorescent light are captured atthe same time by the imaging part 62 in this way, the time required forimaging can be greatly shortened.

Preliminary Verification of Phase Modulation Mask

As described above, the phase modulation mask 50 is configured to becapable of coping with the first fluorescent light and the secondfluorescent light having mutually different wavelengths. Note that phasemodulation masks for appropriately forming an image according to a pointspread function of one type of fluorescent light by generating a phasedifference that is optimum for one kind of fluorescent light aregenerally known. Therefore, in order to make the phase modulation maskcorrespond to two types of fluorescent light, verification is performedby first modulating the phases of the first fluorescent light and thesecond fluorescent light with a phase modulation mask optimal for thefirst fluorescent light, and verification is performed by modulating thephases of the first fluorescent light and the second fluorescent lightby a phase modulation mask optimal for the second fluorescent light.

As shown in FIG. 6 (a), the optical device 10 used in this verificationdiffers from the configuration shown in FIG. 1 in that a polarizingplate 38 and a mirror 39 are provided instead of the beam expander 37,and a phase modulation device 51 is provided instead of the phasemodulation mask 50. The polarizing plate 38 is configured by, forexample, a polarization prism. The polarizing plate 38 is installed suchthat the polarization direction thereof is an appropriate polarizationdirection with respect to the phase modulation device 51. The mirror 39reflects the fluorescent light passing through the polarizing plate 38and guides it to the phase modulation device 51. The phase modulationmask used in this verification is a phase modulation device 51 formodulating the phase when reflecting incident light. The phasemodulation device 51 is disposed on the Fourier plane of the opticalsystem formed by the objective lens 36, the dichroic mirror 35, thepolarizing plate 38, the mirror 39, and the condenser lens 61. Like thephase modulation mask 50 shown in FIG. 1 , the phase modulation device51 has the effect of modulating the point spread function on the imagingsurface 62 a.

As shown in FIG. 6 (b), the phase modulation device 51 includes a liquidcrystal panel 51 a. When the phase modulation device 51 is driven, theliquid crystal molecules 51 b in the liquid crystal panel 51 a rotateaccording to the setting, and the width of the liquid crystal molecules51 b changes in the incident direction of light. When the width of eachliquid crystal molecule 51 b varies in the light incident direction asdescribed above, a difference in refractive index occurs according tothe position in the incident region of the phase modulation device 51.In this way, the phase of the light incident on the liquid crystal panel51 a and reflected by the mirror 51 c is modulated in accordance withthe incident position.

When an image is input, the phase modulation device 51 sets thegradation of each pixel of the liquid crystal panel 51 a based on theinput image. The image input to the phase modulation device 51 holdsinformation representing the gradation of each pixel of the phasemodulation device 51. The phase modulation device 51 acquires thegradation to be set for each pixel from the input image and sets therotation angle of each liquid crystal molecule 51 b so that thegradation of each pixel becomes a desired gradation based on the inputimage. In this manner, the phase modulation device 51 sets the rotationangle of the liquid crystal molecules 51 b based on the input image andsets the gradation pattern of each pixel. Note that in a case in whichthe phase modulation device 51 is configured to be capable of acceptingother than images, the phase modulation device 51 selects each pixel ofthe liquid crystal panel 51 a based on data other than the image holdinginformation representing the gradation of each pixel of the phasemodulation device 51.

In this verification, an “LCOS-SLM 01” manufactured by HamamatsuPhotonics KK was used as the phase modulation device 51. A firstfluorescent bead and a second fluorescent bead were arranged on theslide glass 41 of the stage 40. When irradiated by the light from thelight source 21, the first fluorescent bead generates fluorescent lighthaving a central wavelength of 690 nm, that is, generates a firstfluorescent light. When irradiated by the light from the light source22, the second fluorescent bead generates fluorescent light having acentral wavelength of 530 nm, that is, generates a second fluorescentlight.

In this verification, when observing the first fluorescent light, theobjective lens 36 is scanned in the Z axis direction with respect to thefirst fluorescent bead, and the position of the first fluorescent beadin the Z axis direction is relatively changed. Similarly, when observingthe second fluorescent light, the objective lens 36 is scanned in the Zaxis direction with respect to the second fluorescent bead, and theposition of the second fluorescent bead in the Z axis direction isrelatively changed. By scanning the objective lens 36 in the Z-axisdirection in this way, it is possible to create a state similar to thestate in which a plurality of fluorescent beads are arranged atdifferent positions in the Z-axis direction. Then, the fluorescent lightwas imaged by the imaging part 62, and an image of the fluorescent lightwas acquired for each scanning position of the objective lens 3.6

As shown in FIG. 6 (b), the liquid crystal panel 51 a of the phasemodulation device 51 is configured so that the phase can be modulatedwith 256 gradations per pixel by changing the tilt of the liquid crystalmolecules 51 b. The gradation of each pixel can be set from 0 to 255,and phase modulation of 256 gradations is realized by setting thegradation to any one of 0 to 255. The pattern distribution of thegradation set for all the pixels of the liquid crystal panel 51 a isreferred to as a “phase modulation pattern” hereinafter as adistribution for modulating the phase. That is, the distribution of thegradation set for each pixel of the liquid crystal panel 51 acorresponds to the phase modulation pattern. The gradation of each pixelof the liquid crystal panel 51 a is set by inputting an image to thephase modulation device 51 and setting a phase modulation pattern, andthe fluorescent light entering the phase modulation device 51 changes inphase for each pixel, as shown in FIG. 6 (b).

The first phase modulation pattern shown in FIG. 7 (a) is a phasemodulation pattern optimal for light of the first wavelength, that is,the first fluorescent light. The first phase modulation pattern impartsa first phase modulation to the light of the first wavelength, that is,the first fluorescent light. The second phase modulation pattern shownin FIG. 7 (b) is a phase modulation pattern optimal for light of thesecond wavelength, that is, the second fluorescent light. The secondphase modulation pattern imparts a second phase modulation to the lightof the second wavelength, that is, the second fluorescent light. Whenthe phase modulation pattern of the phase modulation device 51 is set tothe first phase modulation pattern, an image corresponding to the DH-PSFis formed on the imaging surface 62 a for the first fluorescent lightentering the phase modulation device 51. When the phase modulationpattern of the phase modulation device 51 is set to the second phasemodulation pattern, an image corresponding to the DH-PSF is formed onthe imaging surface 62 a for the second fluorescent light entering thephase modulation device 51.

In FIGS. 7 (a) and 7 (b), pixels with a gradient of 0 are shown in blackand pixels with a gradient of 255 are shown in white. Pixels of gradient0 do not modulate the phase of the entering light. The phase of thefirst fluorescent light entering the pixel having the gradient of 255 isshifted by 2π from the phase of the first fluorescent light entering thepixel having the gradient of 0. The phase of the second fluorescentlight entering the pixel having the gradient of 183 is shifted by 2πfrom the phase of the second fluorescent light entering the pixel havingthe gradient of 0.

In this verification and verification to be described later, in order tocorrect the aberration caused by the incident surface of the phasemodulation device 51, a predetermined first correction mask wassynthesized with the phase modulation pattern set in the phasemodulation device 51 when the first fluorescent light enters the phasemodulation device 51, and a predetermined second correction mask wassynthesized with the phase modulation pattern set in the phasemodulation device 51 when the second fluorescent light enters the phasemodulation device 51. By synthesizing the first correction mask for apixel whose gradient exceeds 255, the remaining value obtained bydividing the gradient by 256 was set as the gradient of the pixel. Bysynthesizing the second correction mask for a pixel whose gradientexceeds 183, the remaining value obtained by dividing the gradient by184 was set as the gradient of the pixel.

The results of preliminary verification of the phase modulation devicewill be described with reference to FIGS. 8 (a) to 8 (h).

FIGS. 8 (a) and 8 (b) show the results of verification when the firstphase modulation pattern is set in the phase modulation device 51 andthe first fluorescent light is observed. In this case, as shown in FIG.8 (a), the first fluorescent light is imaged on two points on theimaging surface 62 a, and the straight line formed by the imagingpositions of the two points is rotated 180 degrees by scanning theobjective lens 36 in the Z axis direction. The relationship between thescanning position in the Z axis direction of the objective lens 36 andthe angle of the straight line formed by the imaging positions of thetwo points is a one-to-one corresponding curve as shown in FIG. 8 (b).From the results in FIGS. 8 (a) and 8 (b), it was found that in thiscase an image corresponding to the DH-PSF of the first fluorescent lightcould be properly formed.

FIGS. 8 (c) and 8 (d) show the results of verification when the firstphase modulation pattern is set in the phase modulation device 51 andthe second fluorescent light is observed. In this case, the shape ofDH-PSF collapsed depending on the position in the Z-axis direction asshown in FIG. 8 (c). Further, in the curve showing the relationshipbetween the angle and the scanning position, a large step appears inpart as shown in FIG. 8 (d). From the results in FIGS. 8 (c) and 8 (d),it was found that in this case an image corresponding to the DH-PSF ofthe second fluorescent light could not be properly formed.

FIGS. 8 (e) and 8 (f) show the results of verification when the secondphase modulation pattern is set in the phase modulation device 51 andthe first fluorescent light is observed. In this case, the shape ofDH-PSF collapsed depending on the position in the Z-axis direction asshown in FIG. 8 (e). Also, as shown in FIG. 8 (f), the curve indicatingthe relationship between the scan position and the angle greatlycollapsed, and the scan position and the angle did not correspond one toone. From the results in FIGS. 8 (e) and 8 (f), it was found that inthis case an image corresponding to the DH-PSF of the first fluorescentlight could not be properly formed.

FIGS. 8 (g) and 8 (h) show the results of verification when the secondphase modulation pattern is set in the phase modulation device 51 andthe second fluorescent light is observed. In this case, the two brightspot images become appropriate as shown in FIG. 8 (g). Also, as shown inFIG. 8 (h), the relationship between the angle and the scan position wasa one-to-one curve. From the results in FIGS. 8 (g) and 8 (h), it wasfound that in this case an image corresponding to the DH-PSF of thesecond fluorescent light could be properly formed.

From the results of FIGS. 8 (a), 8 (b), 8 (g) and 8 (h), it was foundthat the combination of setting the phase modulation pattern of thephase modulation device 51 and the center wavelength of fluorescentlight incident on the phase modulation device 51 is optimum in somecases, such that an image corresponding to fluorescent DH-PSF could beproperly formed. This can be said to be appropriate as a result. On theother hand, it was found that the combination of setting the phasemodulation pattern of the phase modulation device 51 and the centerwavelength of fluorescent light incident on the phase modulation device51 is not optimum in some cases, such that an image corresponding tofluorescent DH-PSF could not be properly formed. From this, it was foundthat it is necessary to appropriately select the setting of the phasemodulation pattern of the phase modulation device 51 and the fluorescentlight incident on the phase modulation device 51.

Based on the above results, the inventors considered the integration ofa first phase modulation pattern optimum for the first fluorescent lightand a second phase modulation pattern optimum for the second fluorescentlight so as to correspond to both the first fluorescent light and thesecond fluorescent light. At that time, the inventors focused onoverlapping wavelength bands of the first fluorescent light and thesecond fluorescent light. The first fluorescent light is light having anintensity peak at a first wavelength and the second fluorescent light islight having an intensity peak at a second wavelength. That is, thewavelength band of the first fluorescent light spreads to some extentwith the first wavelength as the center wavelength, and the wavelengthband of the second fluorescent light spreads to certain extent with thesecond wavelength as the central wavelength. Then, a part of thewavelength band of the first fluorescent light and a part of thewavelength band of the second fluorescent light overlap each other.

The inventors have found that when a part of the wavelength band of thefirst fluorescent light and a part of the wavelength band of the secondfluorescent light overlap, an image corresponding to the DH-PSF of thefirst fluorescent light and the second fluorescent light can beappropriately formed by an integrated phase modulation pattern if thefirst phase modulation pattern optimal for the first fluorescent lightand the second phase modulation pattern optimum for the secondfluorescent light are integrated as described below. The phasemodulation patterns of examples 1 to 6 described below are examples inwhich the first phase modulation pattern and the second phase modulationpattern are integrated by various methods. The inventors also verifiedwhether the image corresponding to the DH-PSF of the first fluorescentlight and the second fluorescent light is appropriately formed by thephase modulation pattern of examples 1 to 6.

Phase Modulation Pattern of Example 1

The phase modulation pattern of the example 1 is produced by combiningthe first phase modulation pattern and the second phase modulationpattern at a predetermined ratio for each position. In the case ofsynthesizing the first phase modulation pattern and the second phasemodulation pattern by a:b, the gradient of the phase modulation patternof the first example is calculated based on the first gradient of thefirst phase modulation pattern and the second gradient of the secondphase modulation pattern, as represented in the equation below.Gradient of phase modulation pattern of example 1=(first gradientxa+second gradient xb)/(a+b)(where a and b are both positive real numbers)

Specifically, when using the phase modulation pattern of example 1, animage corresponding to the phase modulation pattern of example 1 isproduced, the produced image is input to the phase modulation device 51,and the phase modulation pattern of example 1 is realized in themodulation device 51 based on the input image. The gradient in eachpixel of the image corresponding to the phase modulation pattern ofexample 1 is set to a gradient between the gradient of light of thefirst wavelength and the gradient of light of the second wavelength.That is, the gradient is calculated by the above equation based on thegradient at the same pixel position of the first phase modulationpattern that is optimal for the first fluorescent light and the gradientat the same pixel position of the second phase modulation pattern thatis optimal for the second fluorescent light. The phase modulationpattern of example 1 is set in the phase modulation device 51, and therotation angle of each molecule 51 b of the liquid crystal panel 51 a isset by inputting the image corresponding to the phase modulation patternproduced in example 1 to the phase modulation device 51.

FIG. 9 is a diagram showing the phase modulation pattern of example 1when (a, b)=(9, 1), (8, 2), (7, 3), (6, 4), (5, 5), (4, 6), (3, 7), (2,8) and (1, 9), respectively. The results of verification of these ninephase modulation patterns are shown below.

FIG. 10 shows the image forming status of the first fluorescent light onthe imaging surface 62 a when the first fluorescent light is observedusing the nine types of phase modulation pattern of example 1 above, thefirst phase modulation pattern, and the second phase modulation pattern.The vertical direction indicates the scanning position of the objectivelens 36. The leftmost “690 nm” and the rightmost “530 nm” indicate thecase of using the first phase modulation pattern and the second phasemodulation pattern, respectively.

As shown in FIG. 10 , at each of the scan positions, a bright spot imagecorresponding to the DH-PSF was observed in a blurred conditionprogressing to the right, that is, as the blending ratio of the secondphase modulation pattern rises and the phase modulation patternapproaches the second phase modulation pattern. Also, when the scanposition is −3 μm or −4 μm progressing to the right, a new bright spotbased on the zero order light appears between the two bright spots. Inparticular, when using the phase modulation pattern of (a, b)=(1, 9) andthe second phase modulation pattern, the intensity of the bright spot ofthe bright spot image of the zero order light was higher compared to thebright spot image corresponding to DH-PSF as shown in the areasurrounded by a solid line frame. In such a case, there is a possibilitythat a curve indicating the relationship between the scan position andthe angle, and a three-dimensional super-resolution image cannot beproperly generated by erroneously recognizing the zero order light asthe bright spot of the DH-PSF.

FIG. 11 shows the image forming status of the second fluorescent lighton the imaging surface 62 a when the second fluorescent light isobserved using the nine types of phase modulation pattern of example 1above, the first phase modulation pattern, and the second phasemodulation pattern.

As shown in FIG. 11 , at each of the scan positions, a bright spot imagecorresponding to the DH-PSF was observed in a blurred conditionprogressing to the left, that is, as the blending ratio of the firstphase modulation pattern rises and the phase modulation patternapproaches the first phase modulation pattern. In particular, when usingthe phase modulation pattern of (a, b)=(9, 1), (8,2), (7,3) and thefirst phase modulation pattern, the number of bright spots was greaterthan two spots depending on the scan position as indicated in the areasurrounded by the solid line frame.

From the results of FIGS. 10 and 11 , it was understood that an imagecorresponding to DH-PSF of both the first fluorescent light and thesecond fluorescent light can be properly formed when the phasemodulation pattern of example 1 was set at (a, b)=(6, 4), (5, 5), (4,6), (3, 7), and (2, 8).

Further, the verification result in the case where the phase modulationpattern of example 1 was set with (a, b)=(5, 5) will be described withreference to FIGS. 12 (a) to 12 (d).

FIGS. 12 (a) and 12 (b) show the results when observing the firstfluorescent light, and FIGS. 12 (c) and (d) show the results whenobserving the second fluorescent light. As shown in FIGS. 12 (a) and 12(c), the image corresponding to the DH-PSF of both the first fluorescentlight and the second fluorescent light can be appropriately formed whenthe slope of the straight line formed by the two imaging positions is inthe range of −90 degrees to +90 degrees. Also, as shown in FIGS. 12 (b)and 12 (d), the relationship between the angle and the scan position wasa one-to-one curve.

From the above verification results, it was found that an imagecorresponding to DH-PSF of both the first fluorescent light and thesecond fluorescent light can be properly formed by the phase modulationpattern of example 1 when the phase modulation pattern of example 1 isset in the phase modulation device 51 if the first phase modulationpattern and the second phase modulation pattern are combined at apredetermined ratio as the phase modulation pattern of example 1. Notethat, similarly for three or more kinds of fluorescent lights havingdifferent center wavelengths, an image corresponding to DH-PSF of thethree fluorescent lights can be appropriately formed by synthesizingoptimal phase modulation patterns for the three types of fluorescentlights at a predetermined ratio.

Next, the synthesis of the first phase modulation pattern and the secondphase modulation pattern as described above will be described in detailwith reference to FIGS. 13 (a) to 13 (c). FIG. 13 (a) through 13 (c)schematically show one pixel position on a liquid crystal panel 51 a inthe first phase modulation pattern, second phase modulation pattern, andthe phase modulation pattern of example 1, respectively. In FIGS. 13 (a)to 13 (c), it is assumed for the sake of convenience that the gradientof one pixel is set by one liquid crystal molecule 51 b.

In order to appropriately form an image corresponding to the DH-PSFbased on the first fluorescent light, the distance by which the firstfluorescent light incident on a predetermined pixel positionreciprocates through the liquid crystal molecules 51 b is set as a firstdistance L1, as shown in FIG. 13 (a). Similarly, in order toappropriately form an image corresponding to the DH-PSF based on thesecond fluorescent light, the distance by which the second fluorescentlight incident on a predetermined pixel position reciprocates throughthe liquid crystal molecules 51 b is set as a second distance L2, asshown in FIG. 13 (b). The magnitude at which the phase of the firstfluorescent light is modulated by the first distance L1 is equal to themagnitude at which the phase of the second fluorescent light ismodulated by the second distance L2. That is, when considering thewavelength of the first fluorescent light, the wavelength of the secondfluorescent light, and the refractive index of the liquid crystalmolecules 51 b, the optical path length of the first fluorescent lightbased on the first distance L1 and the optical path length of the secondfluorescent light based on the second distance L2 are equal to eachother.

In order to appropriately form an image corresponding to the DH-PSFbased on both the first fluorescent light and the second fluorescentlight, the distance by which the first fluorescent light and secondfluorescent light incident on a predetermined pixel position reciprocatethrough the liquid crystal molecules 51 b is set as a third distance L3,as shown in FIG. 13 (c). At this time, the third distance L3 is setbetween the first distance and the second distance. That is, themagnitude of the phase modulation at each position of the phasemodulation pattern of example 1 is set to a magnitude between themagnitude of the phase modulation in the first phase modulation patternand the magnitude of the phase modulation in the second phase modulationpattern. Specifically, similar to the equation of the phase modulationpattern of example 1 described above, the third distance L3 iscalculated by the following equation.Third distance L3=(first distance L1×a+second distance L2×b)/(a+b)

When the phase modulation pattern of example 1 is set, the setting asshown in FIG. 13 (c) is performed at all pixel positions. In this waythe phase modulation pattern of example 1 can appropriately for an imagecorresponding to the DH-PSF of both the first fluorescent light and thesecond fluorescent light.

Note that when (a, b)=(5, 5), the third distance L3 is set between thefirst distance L1 and the second distance L2. In this case, phasemodulation occurs of a magnitude intermediate to the magnitude ofphase-modulating the first fluorescent light by the first phasemodulation pattern and the magnitude of phase-modulating the firstfluorescent light by the second phase modulation pattern. For the secondfluorescent light, phase modulation occurs of a magnitude intermediateto the magnitude of phase-modulating the second fluorescent light by thefirst phase modulation pattern and the magnitude of phase-modulating thesecond fluorescent light by the second phase modulation pattern.

Phase Modulation Pattern of Example 2

As shown in FIG. 14 (a), the phase modulation pattern of example 2 isproduced by disposing the first phase modulation pattern and the secondphase modulation pattern in a mosaic pattern. In the phase modulationpattern of example 2, the region group obtained by dividing the incidentregion into a large number is divided into a first region composed of alight gray region and a second region composed of a dark gray region.Each region of the first region and each region of the second region areadjacent to each other. The first phase modulation pattern is set in thelight gray region, and the second phase modulation pattern is set in thedark gray region. In the phase modulation pattern of example 2, oneregion of the first region and one region of the second region have asquare shape of M pixels on a side.

That is, the phase modulation pattern of example 2 is a phase modulationpattern that includes the region for imparting the first phasemodulation to the light of the first wavelength, that is, the firstregion in which the first phase modulation pattern is set, and a secondregion in which the second phase modulation pattern is set for impartingthe second phase modulation to the light of the second wavelength. Inother words, in a certain region of the incident region within the phasemodulation pattern of the second embodiment, the distance traveled bythe light flux incident on the first region for phase modulation is setto the first distance L1 shown in FIG. 13 (a), and the distance that thelight flux entering the second region travels for phase modulation isset to the second distance L2 shown in FIG. 13 (b).

FIGS. 14 (b) to 14 (d) are diagrams showing the phase modulation patternof example 2 when M=1, 3, and 5, respectively. In FIGS. 14 (b) to 14(d), the arrangement of the first phase modulation pattern and thesecond phase modulation pattern in the vicinity of the center isenlarged and displayed. The results of verification of these three phasemodulation patterns are shown below.

FIGS. 15 (a) to 15 (d) are diagrams showing the results of fluorescentlight observation using the phase modulation pattern of example 2 withM=1. As shown in FIGS. 15 (a) and 15 (b), the two bright spot imagesbecame appropriate, and the relationship between the scan position andthe angle also became a one-to-one curve when observing the firstfluorescent light. As shown in FIGS. 15 (c) and 15 (d), the two brightspot images became appropriate, and the relationship between the scanposition and the angle also became a one-to-one curve when observing thesecond fluorescent light. Therefore, an image corresponding to theDH-PSF of both the first fluorescent light and the second fluorescentlight can be appropriately formed by using the phase modulation patternof example 2 with M=1.

FIGS. 15 (e) and 15 (f) show the results of observing the firstfluorescent light and second fluorescent light using the phasemodulation pattern of example 2 with M=5. In this case as well as in thecase where the first fluorescent light was observed and the case wherethe second fluorescent light was observed, the two bright spot imageswere appropriate. Therefore, an image corresponding to the DH-PSF ofboth the first fluorescent light and the second fluorescent light can beappropriately formed by using the phase modulation pattern of example 2with M=5.

FIG. 16 (a) shows the result of observing the first fluorescent lightwith a wide field of view using the phase modulation pattern of example2 with M=1 and 3. FIG. 16 (b) shows the result of observing the secondfluorescent light with a wide field of view using the phase modulationpattern of example 2 with M=1 and 3. In FIGS. 16 (a) and 16 (b), theupper row is an image generated by the imaging part 62 when fluorescentlight is directly observed. The lower row is the image with the contrastof the upper image adjusted. In FIGS. 16 (a) and 16 (b), the image onthe left side corresponds to the phase modulation pattern with M=1 andthe image on the right side corresponds to the phase modulation patternwith M=3.

In the images of the lower row in FIGS. 16 (a) and 16 (b), bright spotsappear at positions indicated by arrows. The luminescent spot is abright spot based on the diffracted light generated by the first phasemodulation pattern and the second phase modulation pattern beingperiodically arranged in a mosaic pattern. Since the diffraction angleis smaller as the value of M is larger, the diffraction light appears ina region closer to the center of the image in the case of using thephase modulation pattern with M=3 than in the case of using the phasemodulation pattern with M=1. In the case of M=3, it was found that thediffracted light was superimposed on a part of the bright spotcorresponding to the DH-PSF which was desired originally.

Therefore, when using the phase modulation pattern of example 2, it canbe said that it is desirable to make the value of M as small aspossible. However, when the size of one pixel of the phase modulationdevice 51 is small, the diffracted light may not enter the field of vieweven if the value of M is increased. Also, since the bright spot of thediffracted light is darker than the bright spot corresponding to theDH-PSF that is desired, it may not be a problem particularly whencalculating three-dimensional coordinates based on the rotational angle.The value of M that can appropriately form an image corresponding toDH-PSF of two kinds of fluorescent lights having different centralwavelengths is not limited to 1, 3, and 5 as described above.

Phase Modulation Pattern of Example 3

As shown in FIG. 17 (a), the phase modulation pattern of example 3 isproduced by disposing the first phase modulation pattern and the secondphase modulation pattern in a mosaic pattern similar to the phasemodulation pattern of example 2. However, in the phase modulationpattern of example 3, one region of the first region and one region ofthe second region are both rectangular shapes of M pixels in thevertical direction and N pixels in the horizontal direction.

FIGS. 17 (b) to 17 (d) are diagrams showing the phase modulation patternof example 3 when (M,N)=(1,2), (1,4), (1,32), respectively. The resultsof verification of these three phase modulation patterns are shownbelow.

FIGS. 18 (a) to 18 (d) are diagrams showing the results of fluorescentlight observation using the phase modulation pattern of example 3 with(M,N)=(1,2). As shown in FIGS. 18 (a) and 18 (b), the two bright spotimages became appropriate, and the relationship between the scanposition and the angle also became a one-to-one curve when observing thefirst fluorescent light. As shown in FIGS. 18 (c) and 18 (d), the twobright spot images became appropriate, and the relationship between thescan position and the angle also became a one-to-one curve whenobserving the second fluorescent light. Therefore, an imagecorresponding to the DH-PSF of both the first fluorescent light and thesecond fluorescent light can be appropriately formed by using the phasemodulation pattern of example 3 with (M,N)=(1,2).

FIGS. 18 (e) and 18 (f) show the results of observing the firstfluorescent light and second fluorescent light using the phasemodulation pattern of example 3 with (M,N)=(1,32). In this case also thetwo bright spot images were appropriate when observing the firstfluorescent light and when observing the second fluorescent light.Therefore, an image corresponding to the DH-PSF of both the firstfluorescent light and the second fluorescent light can be appropriatelyformed by using the phase modulation pattern of example 3 with(M,N)=(1,32).

FIG. 19 (a) shows the result of observing the first fluorescent lightwith a wide field of view using the phase modulation pattern of example3 with (M,N)=(1,2), (1,4). FIG. 19 (a) shows the result of observing thesecond fluorescent light with a wide field of view using the phasemodulation pattern of example 3 with (M,N)=(1,2), (1,4). In FIGS. 19 (a)and 19 (b), the image on the left side corresponds to the phasemodulation pattern with (M,N)=(1,2) and the image on the right sidecorresponds to the phase modulation pattern with (M,N)=1,4).

In the lower row images of FIGS. 19 (a) and 19 (b), bright spots appearat positions indicated by arrows similar to the case shown in FIGS. 16(a) and 16 (b). The vertical direction and the horizontal direction inFIG. 17 (a) correspond to the horizontal direction and the verticaldirection, respectively, in the images of FIGS. 19 (a) and 19 (b). Inthis verification, since the value of M is fixed at 1, diffracted lightappears at the same position on both ends in the left-right direction asshown in the lower row images of FIGS. 19 (a) and 19 (b). On the otherhand, as the value of N increases, the diffracted light approaches thecenter of the image in the vertical direction while maintaining theposition in the horizontal direction. It is assumed that the diffractedlight approaches the center of the image in the left-right directionwhen the value of M increases from 1 toward 2 as can be seen from theverification of the phase modulation pattern of example 2, even whenusing the rectangular phase modulation patter of example 3 under thecondition of (M,N)=(2,4).

Therefore, when using the phase modulation pattern of example 3, it canbe said that it is desirable to make the value of M,N as small aspossible. Note that the value of M,N that can appropriately form animage corresponding to DH-PSF of two kinds of fluorescent lights havingdifferent central wavelengths is not limited to (1,2), (1,4), (1,32) asdescribed above.

Phase Modulation Pattern of Example 4

As shown in FIG. 20 (a), the phase modulation pattern of example 4 isproduced by disposing the first phase modulation pattern and the secondphase modulation pattern in a stripe pattern. One region of the firstregion and one region of the second region are formed in a stripe shapeof M pixels in the horizontal direction. Each region of the first regionand each region of the second region extend from one end to the otherend of the incident region where the first fluorescent light and thesecond fluorescent light enter. That is, the length in the verticaldirection is set to be the maximum pixel value of the settable length.

FIGS. 20 (b) to 20 (d) are diagrams showing the phase modulation patternof example 4 when M=1, 2, and 16, respectively. The results ofverification of these three phase modulation patterns are shown below.

FIGS. 21 (a) to 21 (d) are diagrams showing the results of fluorescentlight observation using the phase modulation pattern of example 4 withM=1. As shown in FIGS. 21 (a) and 21 (b), the two bright spot imagesbecame appropriate, and the relationship between the scan position andthe angle also became a one-to-one curve when observing the firstfluorescent light. As shown in FIGS. 21 (c) and 21 (d), the two brightspot images became appropriate, and the relationship between the scanposition and the angle also became a one-to-one curve when observing thesecond fluorescent light. Therefore, an image corresponding to theDH-PSF of both the first fluorescent light and the second fluorescentlight can be appropriately formed by using the phase modulation patternof example 4 with M=1.

FIGS. 21 (e) and 21 (f) show the results of observing the firstfluorescent light and second fluorescent light using the phasemodulation pattern of example 4 with M=16. In this case also the twobright spot images were appropriate when observing the first fluorescentlight and when observing the second fluorescent light. Therefore, it isunderstood that an image corresponding to the DH-PSF of both the firstfluorescent light and the second fluorescent light can be appropriatelyformed by using the phase modulation pattern of example 4 with M=16.

FIG. 22 (a) shows the result of observing the first fluorescent lightwith a wide field of view using the phase modulation pattern of example4 with M=1 and 2. FIG. 22 (b) shows the result of observing the secondfluorescent light with a wide field of view using the phase modulationpattern of example 4 with M=1 and 2. In FIGS. 22 (a) and 22 (b), theimage on the left side corresponds to the phase modulation pattern withM=1 and the image on the right side corresponds to the phase modulationpattern with M=2.

In the lower row images of FIGS. 22 (a) and 22 (b), bright spots appearat positions indicated by arrows similar to the case shown in FIGS. 16(a) and 16 (b). However, unlike the case of the phase modulation patternof example 2, diffracted light appeared only in the vertical directionas shown in the lower row images of FIGS. 22 (a) and 22 (b). On theother hand, as the value of M increases, the diffracted light approachesthe center of the image in the vertical direction similarly to the phasemodulation pattern of example 2.

Therefore, when using the phase modulation pattern of example 4, it canbe said that it is desirable to make the value of M as small aspossible. Note that The value of M that can appropriately form an imagecorresponding to DH-PSF of two kinds of fluorescent lights havingdifferent central wavelengths is not limited to 1, 2, and 16 asdescribed above.

Phase Modulation Pattern of Example 5

As shown in FIG. 23 (a), the phase modulation pattern of example 5 isproduced by disposing the first phase modulation pattern and the secondphase modulation pattern in a concentric pattern. In other words, eachregion of the first region and each region of the second region areconcentric ring shapes. Note that the center of each region of the firstregion and the center of each region of the second region also may bedisplaced. One region of the first region and one region of the secondregion are arranged so as to be interchanged at every M pixel intervalfrom the center.

FIGS. 23 (b) to 23 (e) are diagrams showing the phase modulation patternof example 5 when M=1, 2, 5, and 60, respectively. The results ofverification of these four phase modulation patterns are shown below.

FIGS. 24 (a) to 24 (d) are diagrams showing the results of fluorescentlight observation using the phase modulation pattern of example 5 withM=1. As shown in FIGS. 24 (a) and 24 (b), the two bright spot imagesbecame appropriate, and the relationship between the scan position andthe angle also became a one-to-one curve when observing the firstfluorescent light. As shown in FIGS. 24 (c) and 24 (d), the two brightspot images became appropriate, and the relationship between the scanposition and the angle also became a one-to-one curve when observing thesecond fluorescent light. Therefore, an image corresponding to theDH-PSF of both the first fluorescent light and the second fluorescentlight can be appropriately formed by using the phase modulation patternof example 5 with M=1.

FIGS. 24 (e) and 24 (f) show the results of observing the firstfluorescent light and second fluorescent light using the phasemodulation pattern of example 5 with M=5. In this case also the twobright spot images were appropriate when observing the first fluorescentlight and when observing the second fluorescent light. Therefore, it wasunderstood that an image corresponding to the DH-PSF of both the firstfluorescent light and the second fluorescent light can be appropriatelyformed by using the phase modulation pattern of example 5 with M=5.

FIGS. 24 (g) and 24 (h) show the results of observing the firstfluorescent light and second fluorescent light using the phasemodulation pattern of example 5 with M=60. In this case the two brightspot images were collapsed when observing the first fluorescent lightand when observing the second fluorescent light. Therefore, it was foundthat when the value of M becomes too large, it is impossible to properlyform an image corresponding to the DH-PSF of the first fluorescent lightand the second fluorescent light.

FIG. 25 (a) shows the result of observing the first fluorescent lightwith a wide field of view using the phase modulation pattern of example5 with M=2 and 5. FIG. 25 (b) shows the result of observing the secondfluorescent light with a wide field of view using the phase modulationpattern of example 5 with M=2 and 5. In FIGS. 25 (a) and 25 (b), theimage on the left side corresponds to the phase modulation pattern withM=2 and the image on the right side corresponds to the phase modulationpattern with M=5.

In the lower row images of FIGS. 25 (a) and 25 (b), ring-shapeddiffracted light surrounding the bright spot image appears as indicatedby an arrow. Referring to the image of the lower row of M=2 and 5, thisdiffracted light appeared as ring-shaped diffracted light having asmaller diameter as the value of M increases, and approached the brightspot corresponding to the DH-PSF originally desired. Therefore, whenusing the phase modulation pattern of example 5, it can be said that itis desirable to make the value of M as small as possible. The value of Mthat can appropriately form an image corresponding to DH-PSF of twokinds of fluorescent lights having different central wavelengths is notlimited to 1, 2, and 5 as described above.

According to the phase modulation pattern of example 5, the brightnessof the diffracted light is dispersed in a wider range along the circleas compared with the phase modulation pattern of example 2 throughexample 4, so that the luminance of the diffracted light decreases.Therefore, in the case of using the phase modulation pattern of example5, a serious problem does not occur even if the diffracted lightoverlaps the bright spot corresponding to the DH-PSF originally desired.From this, the phase modulation pattern of example 5 in which the valueof M is small differs from the phase modulation pattern of example 2 toexample 4 in that it can be said that an image corresponding to DH-PSFof both the first fluorescent light and the second fluorescent light canbe appropriately formed.

Phase Modulation Pattern of Example 6

The phase modulation pattern of example 6 is produced by arranging thefirst phase modulation pattern and the second phase modulation pattern,in which the positions, sizes and the like are modified, are arranged ina mosaic pattern of a square shape similarly to the phase modulationpattern of the example 2.

FIG. 26 shows eight types of the phase modulation pattern of example 5.“MosaicROTATES” is a phase modulation pattern in which a first phasemodulation pattern rotated 5 degrees around the center and a secondphase modulation pattern are arranged in a mosaic pattern.“MosaicROTATE30” is a phase modulation pattern in which a first phasemodulation pattern rotated 30 degrees around the center and a secondphase modulation pattern are arranged in a mosaic pattern.“MosaicSHIFT5” is a phase modulation pattern in which a first phasemodulation pattern shifted by 5 pixels in the right direction and asecond phase modulation pattern shifted by 5 pixels in the leftdirection are arranged in a mosaic pattern. “MosaicSHIFT10” is a phasemodulation pattern in which a first phase modulation pattern shifted by10 pixels in the right direction and a second phase modulation patternshifted by 10 pixels in the left direction are arranged in a mosaicpattern.

“MosaicEXPAND10” is a phase modulation pattern in which a first phasemodulation pattern has a diameter enlarged by 10 pixels and a secondphase modulation pattern are arranged in a mosaic pattern.“MosaicEXPAND40” is a phase modulation pattern in which a first phasemodulation pattern has a diameter enlarged by 40 pixels and a secondphase modulation pattern are arranged in a mosaic pattern.“MosaicREDUCE20” is a phase modulation pattern in which a first phasemodulation pattern has a diameter reduced by 20 pixels and a secondphase modulation pattern are arranged in a mosaic pattern.“MosaicREDUCE40” is a phase modulation pattern in which a first phasemodulation pattern has a diameter reduced by 40 pixels and a secondphase modulation pattern are arranged in a mosaic pattern. The resultsof verification of these eight phase modulation patterns are shownbelow.

FIGS. 27 (a) and 27 (b) show the results of observation of fluorescentlight using “mosaicROTATE5”. In this case the two bright spot imageswere appropriate when observing the first fluorescent light and whenobserving the second fluorescent light. FIGS. 27 (c) and 27 (d) show theresults of observation of fluorescent light using “mosaicROTATE30”. Inthis case the two bright spot images were collapsed when observing thefirst fluorescent light and when observing the second fluorescent light.Therefore, it was found that when the difference between the rotationangle of the first phase modulation pattern and the rotation angle ofthe second phase modulation pattern becomes large, it is impossible toappropriately form an image corresponding to the DH-PSF of the firstfluorescent light and the second fluorescent light.

FIGS. 27 (e) and 27 (f) show the results of observation of fluorescentlight using “mosaicSHIFT5”. In this case, FIGS. 27 (g) and 27 (h) showthe results of observing fluorescent light using “mosaicSHIFT10” whenobserving the first fluorescent light and when observing the secondfluorescent light. In this case, the two bright spot images wereappropriate in the case of observing the first fluorescent light and thecase of observing the second fluorescent light, but the bright spotimage slightly collapsed as compared with “mosaicSHIFT5”.

Note that the reason the bright point image of “mosaicSHIFT10” isslightly distorted compared to “mosaicSHIFT5” is obvious whenconsidering the case where the fluorescent light of the centerwavelength is incident on the optimal phase modulation pattern for onlyone center wavelength. That is, even if the fluorescent light incidenton the phase modulation pattern is the optimal fluorescent light for thephase modulation pattern, the shape of the bright spot image collapsesas the center of the incident beam moves away from the center of thephase modulation pattern. Therefore, in “mosaicSHIFT10” in which thereis a large shift from the center, the bright spot image tends tocollapse.

FIGS. 28 (a) and 28 (b) show the results of observation of fluorescentlight using “mosaicEXPAND10”. In this case the two bright spot imageswere appropriate when observing the first fluorescent light and whenobserving the second fluorescent light. FIGS. 28 (c) and 28 (d) show theresults of observation of fluorescent light using “mosaicEXPAND40”. Inthis case, the two bright spot images were appropriate in the case ofobserving the first fluorescent light and the case of observing thesecond fluorescent light, but the bright spot image slightly collapsedas compared with “mosaicEXPAND10”.

Note that the reason the bright point image of “mosaicEXPAND40” isslightly distorted compared to “mosaicEXPAND10” is obvious whenconsidering the case where the fluorescent light of the centerwavelength is incident on the optimal phase modulation pattern for onlyone center wavelength. That is, even if the fluorescent light incidenton the phase modulation pattern is the optimal fluorescent light for thephase modulation pattern, the shape of the bright spot image isdistorted as the diameter of the incident beam moves away from thediameter of the phase modulation pattern. For this reason, the brightspot image tends to collapse in “mosaicEXPAND40” that has a large amountof deviation in diameter.

FIGS. 28 (e) and 28 (f) show the results of observation of fluorescentlight using “mosaicREDUCE20”. In this case the two bright spot imageswere appropriate when observing the first fluorescent light and whenobserving the second fluorescent light. FIGS. 28 (g) and 28 (h) show theresults of observation of fluorescent light using “mosaicREDUCE40”. Inthis case, the two bright spot images were appropriate in the case ofobserving the first fluorescent light and the case of observing thesecond fluorescent light, but the bright spot image slightly collapsedas compared with “mosaicREDUCE20”. Note that the reason why the brightpoint image of “mosaicREDUCE40” is slightly distorted compared to“mosaicREDUCE20” is the same reason as the collapse in the case of“mosaicEXPAND40”.

Application to Phase Plate Made of Transparent Member

An example in which the phase modulation pattern shown in example 1 isapplied to a phase plate made of a transparent member will be describedwith reference to FIGS. 29 (a) to 29 (f).

FIG. 29 (a) is a phase plate 91 manufactured so as to correspond to thefirst phase modulation pattern shown in FIG. 7 (a). FIG. 29 (b) is aphase plate 92 manufactured so as to correspond to the second phasemodulation pattern shown in FIG. 7 (b). Phase plates 91 and 92 are madeof the same material as phase plate 52. In phase plate 91, a thickportion corresponds to an area near white in the first phase modulationpattern, and a thin portion corresponds to a region close to black inthe first phase modulation pattern. In phase plate 92, a thick portioncorresponds to an area near white in the second phase modulationpattern, and a thin portion corresponds to a region close to black inthe second phase modulation pattern.

The maximum thickness of the phase plate 91 is designed so that thephase of the first fluorescent light entering the maximum thicknessportion is shifted by one wavelength. Similarly, the maximum thicknessof the phase plate 92 is designed so that the phase of the secondfluorescent light entering the maximum thickness portion is shifted byone wavelength. The phase plate 91 imparts a first phase modulation tothe light of the first wavelength, that is, the first fluorescent light.The phase plate 92 imparts a second phase modulation to the light of thesecond wavelength, that is, the second fluorescent light.

The phase plates 91 and 92 are synthesized in the same manner as thephase modulation pattern of example 1 to produce the phase plate 52 asshown in FIG. 29 (c). The thickness of the phase plate 52 has athickness between the thickness of the phase plate for the light of thefirst wavelength and the thickness of the phase plate for the light ofthe second wavelength, that is, the thickness of the phase plate 52 hasa thickness between the thickness of the phase plate 91 optimal for thefirst fluorescent light and the thickness of the phase plate 92 optimalfor the second fluorescent light. The phase plate 52 is made of atransparent member such as an acrylic resin. Note that the transparentmember configuring the phase plate 52 need not necessarily betransparent, and may be any material as long as it can transmit light.

Note that the thickness T1 of the phase plate 91 and the thickness T2 ofthe phase plate 92 are calculated by the following equations. In thefollowing equations, n1 is the refractive index around the phase plates91 and 92, that is, the refractive index of air. N2 is the refractiveindex of the phase plates 91 and 92, that is, the refractive index ofthe phase plate 52 to be produced. X1 is the center wavelength of thefirst fluorescent light and X2 is the center wavelength of the secondfluorescent light. θ is the phase shift amount.Thickness T1=λ1×θ/{2π(n2−n1)}Thickness T2=λ2×θ/{2π(n2−n1)}Note that T2/T1=λ2/λ1.

For example, in the phase plates 91 and 92, when the maximum shiftamount of the phases of the first fluorescent light and the secondfluorescent light is θmax, the thicknesses T1 and T2 obtained bysubstituting θmax into the above formula correspond to the maximumthickness of phase plates 91 and 92. The thickness of the phase plate 52is set to a thickness between the maximum thickness of the phase plate91 and the maximum thickness of the phase plate 92. Similarly, inregions outside the region where the phase plates 91 and 92 have themaximum thickness, the thickness of the phase plate 52 is set to athickness between the thickness T1 and the thickness T2 obtained basedon the phase shift amount.

In the equation for calculating the thickness, when the range of θ is2(m−1)π<θ≤2 mπ (where m is a positive integer), the following equationis preferable. In this way it is possible to suppress a decrease inlight transmittance.Thickness T1=λ1{θ−2(m−1)π}/{2π(n2−n1)}Thickness T2=λ2{θ−2(m−1)π}/{2π(n2−n1)}

FIGS. 29 (d) to 29 (f) are diagrams schematically showing cross sectionsobtained by sectioning a region surrounded by a dotted line in FIGS. 29(a) to 29 (c) in planes parallel to the thickness direction. Thethickness of the region surrounded by the dotted line is designated afirst thickness H1 in the case of the phase plate 91, and a secondthickness H2 in the case of the phase plate 9. 2 The first thickness H1is a thickness for properly forming an image corresponding to the DH-PSFof the first fluorescent light, and the second thickness H2 is athickness for properly forming an image corresponding to the DH-PSF ofthe second fluorescent light. Here, the phase plate 52 is manufacturedbased on the same method as when manufacturing the phase modulationpattern of example 1, and the thickness of the phase plate 52 isdistributed with thickness between the thickness of the phase plate 91and the thickness of the phase plate 92. Therefore, when the thicknessof the phase plate 52 in the region surrounded by the dotted line isdesignated a third thickness H3, the third thickness H3 is calculated bythe following equation.Third thickness H3=(first thickness H1×a+second thickness H2b)/(a+b)

As in the case where the phase modulation pattern of example 1 is set inthe phase modulation device 51, the phase plate 52 manufactured in thismanner properly forms the image corresponding to the DH-PSF of both thefirst fluorescent light and the second fluorescent light. As in theverification results in the phase modulation pattern of example 1, thecloser are the value of a and the value of b, the more properly theimage corresponding to the DH-PSF of both the first fluorescent lightand the second fluorescent light can be formed.

An example in which the phase modulation pattern shown in example 2 isapplied to a phase plate made of a transparent member will be describedwith reference to FIGS. 30 (a) to 30 (i).

The phase plate 91 of FIG. 30 (a) is the same as the phase plate of FIG.29 (a), and the phase plate 92 of FIG. 30 (b) is the same as the phaseplate of FIG. 29 (b). The phase plates 91 and 92 are arranged in amosaic pattern the same as the phase modulation pattern of example 2 toproduce the phase plate 52 as shown in FIG. 30 (c).

As shown by light gray in FIG. 30 (d), the phase plate 91 is fabricatedso as to have an optimum thickness only for the first fluorescent lightthroughout the entire region. As shown by light gray in FIG. 30 (e), thephase plate 92 is fabricated so as to have an optimum thickness only forthe second fluorescent light over the entire region. That is, the phaseplate 91 is set similarly to the first phase modulation pattern, and thephase plate 92 is set in the same manner as the second phase modulationpattern. As indicated by the mosaic pattern of light gray and dark grayin FIG. 30 (f), the phase plate 52 shown in FIG. 30 (c) has a firstregion of optimal thickness for the first fluorescent light and a secondregion of optimal thickness for the second fluorescent light arranged ina mosaic pattern.

In other words, the first region indicated by light gray in FIG. 30 (f)is a region configured to impart the first phase modulation to the lightof the first wavelength, that is, the first fluorescent light, in theincident region. The second region indicated by dark gray in FIG. 30 (f)is a region configured to impart the second phase modulation to thelight of the second wavelength, that is, the second fluorescent light,in the incident region.

FIGS. 30 (g) to 30 (i) are diagrams schematically showing cross sectionsobtained by sectioning a region surrounded by a dotted line in FIGS. 30(d) to 30 (f) in planes parallel to the thickness direction.Hereinafter, for convenience, the value of M in the phase modulationpattern of example 2 is set to 1. The thickness of the four pixelsection of the region surrounded by the dotted line is a first thicknessH11 to H14 in the case of the phase plate 91, and a second thickness H21to H24 in the case of the phase plate 92. In this case the thickness ofthe phase plate 52 becomes, for example, a thickness that alternatinglyappears as the thickness of the phase plate 91 and the thickness of thephase plate 92, as shown in FIG. 30 (i).

In other words, the configuration of the phase plate 52 shown in FIG. 30(i) is as follows. The phase plate 52 includes a first region composedof a light gray region and a second region composed of a dark grayregion in the incident region, as shown in FIG. 30 (f). Each region ofthe first region and each region of the second region are adjacent toeach other. The same thickness as that of the phase plate 91 is set inthe first region. The same thickness as that of the phase plate 92 isset in the second region. As shown in FIG. 30 (i), the thickness at theposition corresponding to the first region is designated the firstthickness H11 and H13, and the thickness at the position correspondingto the second region is the second thickness H22 and H24.

As in the case where the phase modulation pattern of example 1 is set inthe phase modulation device 51, the phase plate 52 manufactured in thismanner properly forms the image corresponding to the DH-PSF of both thefirst fluorescent light and the second fluorescent light.

Note that although the structural example of the phase modulation mask50 shown in FIG. 1 has been described in terms of the phase plate 52made of a transparent member and phase modulation device 51 having aliquid crystal panel 51 a, the structure of the phase modulation mask 50is not limited to this example. For example, the phase modulation mask50 also may be a deformable mirror provided with minute mirrors thatreflect light according to the setting at different positions in theincident direction at each position within the incident surface.Alternatively, the phase modulation mask 50 also may be a reflectionmember that reflects light at different positions in the incidentdirection at each position in the incident surface.

What is claimed is:
 1. A method for forming images according to a pointspread function, the method comprising: directing light of a firstwavelength having a center wavelength at λ1 and light of a secondwavelength having a center wavelength at λ2 to a phase plate to modulatethe phases of the light, and collecting phase-modulated light of thefirst wavelength and phase-modulated light of the second wavelength toform images respectively for each light of wavelength, wherein athickness T3 of the phase plate is set between a first thickness T1 anda second thickness T2, each represented by the following equations:T1=λ1×θ/{2π(n2−n1)}T2=λ2×θ/{2π(n2−n1)}T3={(T1×a)+(T2×b)}/(a+b) wherein: a and b are both positive realnumbers, T1 is an optimal thickness of a phase plate to form an image oflight at the first wavelength, T2 is an optimal thickness of a phaseplate to form an image of light at the second wavelength, n1 is arefractive index of air, n2 is a refractive index of the phase plate, θis a phase shift amount, and wherein T3 is set within a range where (a,b)=(2, 8) to (8, 2).
 2. A method for forming images according to a pointspread function, the method comprising: irradiating a sample with lightof a first wavelength and light of a second wavelength; directing afirst fluorescent light corresponding to the first wavelength and asecond fluorescent light corresponding to the second wavelength from thesample to a phase plate to modulate the phases of the first and secondfluorescent light, the phase plate made of a transparent material andbeing transparent and comprising a geometry hybrid of a plurality offirst phase modulation regions for the first fluorescent light and aplurality of second phase modulation regions for the second fluorescentlight, wherein the first phase modulation regions and the second phasemodulation regions are arranged in a mosaic, and the first and secondfluorescent lights are directed to pass the first phase modulationregions and the second phase modulation regions at the same time;collecting phase-modulated first fluorescent light and phase-modulatedsecond fluorescent light to an imaging device obtaining a first image ofthe phase-modulated first fluorescent light by the imaging device; andobtaining a second image of the phase-modulated second fluorescent lightby the imaging device.
 3. The method of claim 2, wherein each of thephase-modulated first fluorescent light and the phase-modulated secondfluorescent light is collected to focus at two focal points on a planeof an imaging sensor.
 4. The method of claim 3, further comprisingcapturing the first image and the second image formed on the plane withthe imaging sensor.
 5. The method of claim 3, further comprising:capturing the first image and the second image formed on the plane withthe imaging sensor; and digitally processing the first image and thesecond image to analyze an object, wherein the first fluorescent lightand the second fluorescent light are fluorescent light derived from asubstance to be examined as the object.
 6. The method of claim 2,wherein the geometry of the phase plate is configured such that theplurality of first phase modulation regions and the plurality of secondphase modulation regions are arranged alternately.
 7. The method ofclaim 6, wherein plurality of first regions corresponds to a first phasemodulation pattern and the plurality of second regions corresponds to asecond phase modulation pattern.
 8. The method of claim 2, whereinshapes of the first and second regions are rectangular.
 9. The method ofclaim 2, wherein shapes of the plurality of first regions and theplurality of second regions are ring.
 10. The method of claim 2, whereina wavelength band of the first fluorescent light and a wavelength bandof the second fluorescent light each have a spread and a part of thewavelength band of the first fluorescent light and a part of thewavelength band of the second fluorescent light overlap each other. 11.The method of claim 2, further comprising: capturing a plurality oftwo-dimensional images for the first fluorescent light and a pluralityof two-dimensional images for the second fluorescent light over time.12. The method of claim 11, further comprising: generating athree-dimensional image from the plurality of two dimensional images forthe first fluorescent light and the plurality of two dimensional imagesfor the second fluorescent light.
 13. The method of claim 11, wherein aprocedure of directing, collecting and capturing for the firstfluorescent light and a procedure of directing, collecting and capturingfor the second fluorescent light are carried out in parallel.
 14. Themethod of claim 11, wherein a procedure of directing, collecting andcapturing for the first fluorescent light and a procedure of directing,collecting and capturing for the second fluorescent light are carriedout in different time frames.
 15. A method for forming images accordingto a point spread function, the method comprising: irradiating a samplewith light of a first wavelength and light of a second wavelength;configuring a phase modulation mask to represent on its surface a hybridphase modulation pattern of a plurality of first phase modulationregions for the light of the first wavelength and a plurality of secondphase modulation regions for the light of a second wavelength; directinga first fluorescent light corresponding to the first wavelength and asecond fluorescent light corresponding to the second wavelength from thesample to the phase modulation mask to modulate the phases of the firstand second fluorescent light with the hybrid phase modulation pattern,wherein the plurality of first phase modulation regions and theplurality of second phase modulation regions are arranged in a mosaic,and wherein the first and second fluorescent lights are directed to passthe first phase modulation regions and the second phase modulationregions at the same time; collecting phase-modulated first fluorescentlight and phase-modulated second fluorescent light to an imaging device;obtaining a first image of the phase-modulated first fluorescent lightby the imaging device; and obtaining a second image of thephase-modulated second fluorescent light by the imaging device whereinthe plurality of first phase modulation regions and the plurality ofsecond phase modulation regions are arranged in a mosaic.
 16. The methodof claim 15, wherein the phase modulation mask comprises a crystalliquid panel, a gradient of each pixel of which is adjusted in responseto an input.
 17. The method of claim 15, wherein the hybrid phasemodulation pattern is configured by segmenting a plane of the phasemodulation mask into a plurality of regions including first regions andsecond regions, and allocating the first region and second region agradient corresponding to the plurality of first phase modulationregions and the plurality of second phase modulation regions,respectively.